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PREFACE 

This  Memorandum  points  out  the  connection  between  some 
existing  methods  of  factor-analysis  which  do  not  have  an  associated 
communality  problem  and  the  use  of  instrumental  variables  in 
econometric  errors-in-variables  models.  It  should  be  of  particxilar 
interest  to  psychometricians,  and  of  general  interest  to  econo¬ 
metricians  and  statisticians. 


SUMMARY 


The  factor-«nalysls  model  is  rewritten  as  a  system  of  linear 
structural  relations  with  errors  in  variables .  The  method  of 
instrumental  variables  is  applied  to  this  revised  form  of  the  model 
to  obtain  estimates  of  the  factor  loading  matrix.  The  relation 
betv/een  this  method  and  interbattery  analysis,  proportional  profile 
analysis,  and  canonical  factor-analysis  is  pointed  out.  In  addition, 
an  estimation  procedure  based  on  replicated  sampling  different  from 
propoz^on^  profile  ansilysis  is  given. 
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INSTRUMENTAL  VARIABLES  IN  FACTOR  ANALYSIS 


1.  INTRODUCTION 

The  factor  analysis  model  [2]  is  given  by 

(1)  Z  ■  FX  +  U, 

where  Z  is  a  vector  of  n  components  (n  test  scores,  say),  X  is 
a  vector  of  r(<  n)  components  (the  common  factor  scores),  U  is 
an  n-^ector  (the  unique  part  of  the  test  scores)  and  F  is  an  n  x  r 

matrix  (of  factor  loadings).  It  is  assumed  that  U  and  X  are  - 

independent  random  vectors,  with  ^U  =  ^X  »  0,  £^UU'  ®  D, 

a  diagonal  matrix,  and  ^(XX')  »  M.  From  these  assumptions, 
one  deduces  that  the  covariance  matrix  C  of  Z  is  given  by 

(2)  L  -  ^ZZ'  »  FMF'  +  D. 

After  assuming  that  M  -  I,  the  identity  matrix,  the  usual 
attack  in  determining  F  (or  estimating  F  if  C  is  unknown  and 
only  estimated)  is  to  proceed  from  this  equation,  or  equivalently 
from  the  equation  corresponding  to  (2)  based  on  the  correlation 
matrix  of  Z.  Certainly  if  D  were  known,  the  problem  of  finding 
F  would  be  trivial,  as  it  is  given,  up  to  rotation,  by  "factoring"  the 
matrix  Z— D.  Thus,  the  usual  factor-analytic  techniques  can  be 
characterized  as  attempts  either  at  successively  better  approximating 
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D  (or  equivalently  the  diagonal  terms  of  FF',  i.e.,  the  com— 
munalities)  or  at  assuming  away  the  problem  of  estimating  D  by 
imposing  additional  restrictions  on  the  problem  which,  in  addition, 
eliminate  the  indeterminancy  of  F  due  to  rotation  (for  example,  the 
restriction  that  F'D  ^F  be  diagonal,  as  in  Rao's  work  ([  16] ). 

In  this  Memorandum  we  shaill  instead  proceed  directly  from 
eq::ation  (1)  to  the  problem  of  estimating  F.  We  shall  see  that  the 
problem  is  equivalent  to  that  of  fitting  a  linear  relation  when  both 
the  independent  and  dependent  variables  are  subject  to  error,  as  has 
been  well  known  to  some  factor  analysts,  notably  Burt  [3]  (see 
also  [6] ),  and  thus  that  the  difficulty  in  estimating  F  without  some 
additional  restrictions  on  the  problem  (such  as  Rao's  alluded  to 
above)  is  just  a  reflection  of  the  difficulties  in  the  equivalent  prob¬ 
lem  (see  [  14]  for  a  catalogue  of  these).  We  shall  in  particular  look 
into  one  of  the  approaches  taken  by  econometricians  when  confronted 
with  the  "errors-in-variables"  problem,  namely  the  use  of  in— 
strumental  variables,  and  see  that  in  essence  this  approach  is  what 
is  being  taken  by  Tucker  and  Gibson  ([  9] ,  [lO] ,  [  II]  ,  and  [  20] ) 
in  the  interbattery  method,  but  that  they  are  working  with  equation 
(2),  in  the  tradition  of  factor  analysts,  rather  than  with  equation  (1). 
We  shall  then  point  out  connections  between  this  approach  and  that 
of  proportional  profile  analysis,  analysis  of  variance  in  factor 
ana^sis,  and  canonical  factor  analysis. 


2.  THE  "ERRORS-^N-VARIABLES"  MODEL 


As  the  rank  of  F  is  r,  there  is  some  r  x  r  submatrix  of 
F  which  is  nonsingular.  Suppose  for  convenience  that  it  is  the 
submatrix  cmsisting  of  the  first  r  rows.  Write 


■^(1) 

"(1) 

z  - 

m 

X  + 

.^<a. 

''(2) 

"(2) 

where  and  Ujjj  have  r  rows.  Then 


or 

Thus 


Let  B  *  ^^2)  ^(iV  above  equation  is  a  linear  structural 

relation  between  the  "true"  variables  •  Z^^^^  “^(1) 

and  Z^2)  *  Z^2)  where  the  respective  true  variables  are 

c^served  with  "error"  and  U^2)'  ^  ^  is  unique 

up  to  a  right-multiplication  by  a  nonsingular  matrix,  we  might  as 
well  adopt  as  the  canonical  form  for  the  matrix  F  the  form 


^(1)  *  ^(1)  ^  ^(1) » 


,-l 


^(2)  *  ''(2)  ^  ^  ^(2)  ’  '’(2)  ®’(1)^^(1)  “^(1)^  ^  ^(2) 


for  right-multiplication  by  the  nonsingular  matrix  yields  the 
originiil  matrix  of  interest,  F.  Our  problem  is  then  to  estimate 
B. 

One  should  parenthetically  note  that  much  of  the  early  dis¬ 
cussion  of  the  identifiability  of  the  linear  structural  relation  with 
errors  in  variables  (  [15] ,  [17],  [19])  was  stimulated  by  and 
dlrSetAd  the  factoi*— unlvala  model  -as- tn  this  light. - 


3.  INSTRUMENTAL  VARIABLES  AND  INTERBATTERY  ANALYSIS 


Suppose  now  that  n  —  r  >  r,  and,  in  addition  to  making 
observations  on  the  n  items  comprising  Z,  we  also  observed  a 
vector  of  B>r  additionsd  items,  where 

?2,3,-  gu,„u;„  IB  a  diagonal  matrix,  is 


'(3) 


'(3)  ''(3) 


(3) 


uncorrelated  with  U,  and  Z*,  and  Z^g^  is  uncorrelated 

-with-XL^  ItLeffert,  we  aHBume  that  our  vector  of  test  items  is  en¬ 


larged  to  dimension  n  s,  and  that  the  usual  factor-analytic 
assumptions  hold  for  this  enlarged  vector  of  test  items.  Let 


^3) 


SO  that 


^(3)  •  *'(8)  ®(i)  <^(i)  “^(1)^ 


p 


•nd  0  •  .  llMn 


Zf.K  O'. 


h 


Thus,  if  the  cross-covariance  matrix  of  with  is  of 
maximum  rank  r,  then 

B  ■  If  Z^2)  ^(3)^  ^^^(3)  ^(i)J  ^8^(1)  ^ja)^  ^^^(3)^(1)^ 


-I 


Also 


^2(2)% 


so  that  if  the  cross-covariance  matrix  of  Z^^^  ^ 

maximum  rank  r,  then 


(1)  "(2)^  ^^“(2)  “(3)^ 


The  vector  Z^^^  plays  the  same  role  in  this  analysis  as  does 
the  "instrumental  variable"  ([7] ,  [18] )  in  the  "errors-inrvariables" 
model.  Notice  that  the  instrumental  variable  satisfies 


if  and  only  if  &  -  B  under  the  assumptions  about  rank  of  cross- 
covariance  matrices  made  earlier.  This  can  in  fact  be  taken  as  the 
definition  of  an  instrumental  variable,  namely  a  vector  of  items 
which  is  uncorrelated  with  the  error  variable  —  BU^j. 

Notice  that  in  our  canonical  form  the  test  items  comprising 
form  a  basis  for  our  r-dimensional  factor  space.  The  matrix 
B  is  the  matrix  of  a  linear  transformation  mapping  x*— space  onto 
(n—r)— space.  Given  n  +  s  test  items,  we  can,  in  a  fsishion  dual 
to  our  interpretation  of  the  notion  of  instnunental  variable,  inter¬ 
pret  the  matrix  B  of  factor  loadings  as  follows.  It  is  the  matrix 
of  that  linear  transformation  whose  images  in  (n— r)-space 

(which  we  might  call  "pseudo-factors")  are,  for  the  purpose  of 
telling  us  the  correlation  between  factors  in  (n—r)— space  and  factors 
in  s-epace,  as  good  as  the  true  factors  in  (n— r)-epace. 

Now  let  us  make  the  identification  of  the  vector  Z  with 
battery  1,  with  subbattery  x,  Z^^^  with  subbattery  y,  and 

-With battery  2,  in  Gibson's  terminology  [ll]  .  We  see  from 
this  that  the  interbattery  method  of  factor  analysis  "works"  because 
it  views  the  more  than  3r  observed  test  items  in  the  manner  in^ 
dicated  above,  as  three  sets  of  items,  one  a  basis  for  the  factor 
space  and  the  other  two  as  instrumental  variables  relative  to  each 
other.  From  our  point  of  view,  the  matrix 


of  factor  loadings  is  as  good  a  mathematical  solution  of  the  problem 
as  any  other.  However,  there  is  concern  over  obtaining  orthogonal 
factor  matrices  in  psychometric  work.  The  technique  of  Gibson 
[11]  as  applied  to  our  canonical  form  will  produce  orthogonal  factor 
matrices. 

The  connection  between  this  development  and  the  work  of 

A^ertXl]  should  be  noted.  In  ][  l]  -it-ia^shown4hat^8ufficient - 

condition  for  the  identifiability  of  the  factor  matrix  (up  to  a  linear 
transformation)  is  that  the  number  of  items  is  at  least  three  times 
the  rank  of  the  factor  matrix  r  and  that  the  items  can  be  partitioned 
into  three  sets  such  that  the  cross-covariance  matrices  between  the 
vectors  of  each  of  the  sets  be  of  maximum  rank  r. 

Finally,  one  should  note  that,  given  N  independent  observations 
fZ^l),  i-1,  ...,  N]  on  Z^^j,  Z^^^,  and  Z^g^,  one 

"clm  estii^ie  the  cross-covariance  matrix  between  Z^^^  and 
o  ^  a,  ^  «  1,  2,  3,  by 

I 

N  ' 
i»l  ” 


and  so  use  of  these  estimates  in  place  of  the  ^  Z|^^  in  the 

above  eqtiations  yields  a  consistent  estimate  of  the  factor  matrix. 


4.  PROPORTIONAL  PROFILES 


Suppose  we  administer  our  n-item  test  to  another  independent 
random  sample  of  size  N .  Let  denote  the  vector  of  scores  of 
the  i*^  individual  of  the  new  sample,  i  ■  1,  . . . ,  N.  Let 

Z  -  F  X  +U 


^(3)  ’  ^(3)  ^  ^(3)  » 


where  F  and  F^^^  are  nxr  matrices  and  F^^^  ■  FA ,  in  which  A 


is  a  diagonal  r  x  r  matrix.  The  i-th  diagonal  element  of  A  is 
the  differential  degree  of  selection  of  the  i-th  fact  or  between  samples 
1  and  2.  This  model  is  Cattell's  proportional  profile  model  [  4] , 


Now  write  the  model  in  Z  in  canonical  form,  so  that 


2(2)-S<^(1)-V"^(2) 


(3)  B 


One  can  thus  treat  the  replications  as  a  vector  of  instrumental 
variables  with  s  *  n  and  G  »  j  A.  Using  the  results  of  Sec.  3, 
one  can  obtain  estimates  of  B  and  Q,  and  thus  of  A,  by  replacing 
B  and  Q  in  the  equation 


A  •  +  [IB]G 


by  their  sample  estimates,  where  1  is  the  r  x  r  identity  matrix. 
This  uses  more  of  the  data  in  estimating  A  than  does  the  use  of 
just  the  upper  r  x  r  submatrix  of  the  estimate  of  G.  Once  one 
has  this,  one  can  obtain  orthogonal  factor  matrices  using  Gibson's 
result  [8] . 


S.  REPUCATED  SAMPLING 

An  alternative  model  in  which  to  utilize  the  replications  of 
the  proportional  profile  model  is  to  assume  that  «  1  and  to 
analyze  the  data  in  a  vector  analysis  of  variance.  For  this  analysis, 
let  and  ^  denote  the  replications  of  and 

Z<2)-  Let 


and 


.-1,2,  1-1 . H, 


- 

-  E  zL./N,  «-1,  2. 

i-1  '  ' 


We  can  now  look  at  the  following  vectors  of  ai^ropriate 
deviaticnis: 


a  •  1,  2,  i  ■  1,  . . ,  N, 
i  •  1,  ...,  N,  «  ■  1,  2,  j  •  1, 


•1(^ 


Then 


is  an  unbiased  estimate  of  the  covariance  matrix  of  U.  v,  a  ■  1,  2  , 
N  '  ^ 

2  L  Cjj  C^j/(N— 1)  is  an  unbiased  estimate  of  +  2  ^fl)’ 

N 

2  L  C^^/(N— 1)  ■  ^  ^  unbiased  estimate  of 

i»l 

N 

2  2  bCz*jj  Z*^y  2  E  Cgj  C^j/(N-1)  i,  an  unbiased 

Vstimale  of  +  2  Zf ^  +  2Bt  Zfi,  - 

Thus  B  can  be  consistently  estimated  from  these  data  in  a  variety 
of  ways.  The  simplest  of  these  is  to  estimate  t  ”  ^11 


rc„o;j/<N-i)-D  /2.S„ 

i»l 


next  ^2?i)^*2)“®12  ^12'  estimating  B  by 


^  A  _1 

®-Si2Sn  • 


The  fact  that  the  proportional-profiles  model,  where  A  ■  I, 
and  so  where  the  degrees  of  selection  are  not  at  all  different,  can 
be  analyzed  is  quite  contrary  to  the  forebodings  of  Gibson  ([8] , 
p.  136).  In  fact,  the  development  in  this  sectimi  can  be  extended  to 
arbitrary,  but  known,  A. 


6.  CANONICAL  AND  INSTRUMENTAL  VARIABLES 

Harris  [12]  and  Kaiser  [13]  have  presented  the  following 
intuitive  picture  of  the  basis  for  Rao's  canonical  factor  analysis. 
Consider  the  random  (n-fr)-dimeneional  vector 


It  has  covariance  matrix 

L  FM 
MF'  M 

and  the  squared  canonical  correlations  between  Z  and  X  are  given 
by  the  roots  of  the  determinants!  equation 

Ifmf*  -  X  E[  «  0  • 

Since  L  ■  FMF'  +  D,  this  is  equivalent  to  finding  roots  of  the 
equation 

Ipmf' -qd\  -  0  -  Ie  +  D-X  e|  •  1e-  e  d|  , 

where  ©■X/(l— X)  and  i  »  —  1/(1  — X)  ■  — (1  +  0).  Let  U  be 
the  matrix  of  characteristic  vectors  associated  with  the  roots  O, 
and'  let  0  be  the  diagonal  matrix  of  0's,  where 


12- 


Then 


(D"^^  ^  PMP*  ^)U  -  U©  . 
FMP‘  UeU‘D^^^ 


and,  up  to  a  linear  transformation, 


P  « 


If  M  *  I,  then  this  is  correct  up  to  an  orthogonal  trsmsformation, 

—1  -  -  - - -  .  _ _ 

and  if  F'D  F  is  diagonal,  then  U  must  be  orthogonal  and  tbe 

above  factor  loadings  are  unique. 

Let  us  look  for  a  moment  at  what  would  be  the  perfect 
instrumental  variable.  As  noted  in  Sec.  3,  an  instrumental  variable 
is  one  which  is  uncorrelated  with  —  B  U^j.  Certainly  X, 
which  is  imcorrelated  with  the  U's,  fits  this  description.  And  the 
Harris-Kaiser  description  of  canonical  factor  analysis  points  up  that 
X  can  be  viewed  initially  as  an  additional  set  of  variables,  i.e. ,  as 
an  instrumental  variable. 

The  difficulty  with  using  X  «  in  the  equations  of  Sec.  3 

is  that  X  is  not  observable.  This  difficulty  is  circumvented  hi 
canonical  factor  analysis  by  the  assumption  that  F'D  ^F  is 
diagonal.  What  are  the  implications  of  this  assumpthm  <m  our 
canonical  form  of  the  factor  analysis  problem? 


Since  F*  «  (I  B')f  we  see  that 


F  «d“^  F  -  d“^  +  B‘D“^  B  ■  a  , 
eay.  We  also  have  the  equation 


L  D 


•1 


P  •  F  A 


9 


Since  Ej  ■  I  +  D^,  ■  BB*  +  D^,  and  »  B*,  the  equation 

A  .  d7‘  +  B-  d;*  B  •  E,  d7‘  +  Ej  d7‘  B 

-  (I  +  Dj)  +  B'  d7‘  B 

holds  only  if  ■  0,  so  that  has  no  meaning. 

Also , 


BA  ■  BD~^  +  BB*  B  ■  ^4  ® 


holds  only  If  B  ■  0.  Thus  when  the  sssumptlons  of  canonical 
factor  analysis  hold,  we  see  this  immediately  in  our  canonical 
form,  as  then  the  first  r  items  are  the  factors  and  are  observed 


without  error. 
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